posted 01-07-2009 01:30 PM            

Here's the problem: Numerically I have a NDI. Polyscore tells me it's DI and OSS-3 tells me "No Significant Reactions" with a
probability of less than .001 of deception.

Anyone encounter anything similar?

posted 01-07-2009 02:32 PM            

Like plane crashes, these things can be studied and understood. There is generally a reason for everything, and we learn from studying these interesting cases.

CK sent me an interesting set of charts for which he had concerns, and we (not me, but the programmers) identified and fixed a software bug in the way that Kircher measurments are obtained in the Lafayette OSS tool - which caused rare measurement inaccuracies. Expect an updated software release soon.

Look at C3 and R3 on chart 1, on these charts.
http://www.polygraphplace.com/cgi-bin/ubbcgi/forumdisplay.cgi?action=displayprivate&number=11&topic=000877

Then look at the table of measurements.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 01-07-2009 04:11 PM            

if you, the examiner of record, score it plus 5 at both spots, then it is NDI/NSR.

I have little to no faith in the polyscore program other than as an interrogative prop (which works great for the younger set, as they blindly believe in the ability of a computer). OSS, may be different, I do not know it, but I am not willing to blindly accept a computer algorythm over my evaluative abilities.

There have simply been too many times that polyscore has been opposite of my calls. Besides, I believe there is an ethical issue in using or deferring to algorythms for test results. (garbage in/garbage out)!

posted 01-07-2009 06:04 PM            

Jim S.,

posted 01-07-2009 08:42 PM            

The following is based on your declarative statements:

I find it interesting that Polyscore has not been validated against what I believe is the most valid polygraph technique in use today, the Bi-zone/You Phase format.

This being the case, I am interested in knowing what testing formats have Polyscore been validated against? Any? Maybe I need to change my formats... Also, I have heard of many examiners who use Polyscore to give them "the answer"; and this, to the exclusion of their own data evaluation - that is, if they even bother to make their own evaluation. This is a very dangerous and slippery slope for our profession.

Further, while I have no knowledge or information about OSS (though I have seen all the fanaticism surrounding its introduction and promotion), I do not believe any computer algorithm is more accurate than an examiner (of record) in field use, during actual specific issue testing. Do you have any independent research to support your claim to the contrary? Also, you wrote, "Since it does outperform most examiners", please present your support, as I would find that claim hard to support (though not necessarily difficult to believe in some circumstances/examiners I have witnessed). Additionally, have "most examiners" been tested against it with their existing accuracy and ability? I for one, was never contacted to participate in that experiment/research.

And, as for your challenge. Please feel free to establish the parameters for such a "contest." If OSS is more accurate than I in actual field testings, artifacts, movements, abnormalities, etc then I will push to support its use. What do you propose in regards to your challenge or ability?

Lastly, my position is that polygraph instrumentation is a business, like many others. The difference between auto mechanics and our instrumentation, for example, is that the basics of auto mechanics and motors have in-deed changed drastically over the years, but polygraph instrumentation (after 100 years) is still three physiological channels and looking for "bumps"...

Why would anyone WANT to use any algorithm to determine the result of an examination?
All of this seems to me to be no more than another business gimmick to introduce and help weak examiners be more confident, sell more product and attempts to make polygraph akin to rocket science, only understood by the elite, enlightened few...

I look forward to your response,

Jim

posted 01-07-2009 09:33 PM            

Read your current APA journal.

Initially, OSS-3 went up against 10 virtually hand-picked scorers who were believed to be very good at polygraph. (The scorers scored field tests - for paired testing certification -in which ground truth was known.) Of the 10 scorers, OSS-3 outperformed 9, and I don't think there was a significant difference between the top scorer and OSS-3. All you have to do to accept the "challenge" is score 100 of the charts in the paired testing sample. That's been an invitation that's been out to all here for quite some time.

(The Utah algorithm outperformed hand scorers when it was tested too - all the way back in the 80s.)

In regard to Polyscore, you can read it's very short manual and see what it was validated on. You can use it for a Bi-zone, but Polyscore says it requires four charts - not three as was used here. (Although, I don't think that would make much difference, but who knows?)

There was an Israeli study on the Backster version of the Bi-zone. It wasn't any more accurate than any other, which has been, Honts, for example, position for some time: a good CQT with a couple of well-developed CQs and a couple of well-developed RQs, then you've got a good CQT that should have an accuracy in the neighborhood of other good CQTs. I haven't read it yet, but there is a study published on the AF MGQT 2-RQ / 3-CQ version in the current edition of Polygraph too.

OSS-3 is free. If anything, they've lost money.

As far as anyone wanting to use an algorithm to determine the result of an exam, there are many sound reasons. For one, you have perfect reliability.

OSS-3 is relatively easy to understand. It measures three features: RLL and amplitude increase in the cardio and EDA. All you have to do is make sure the data is good enough to score, and it'll get you the right answer over 90% of the time. (And yes, I've disagreed with it before - and got a confession.) The math would be difficult for many, sure, but all it does is weight the components differently - something hand-scorers don't do even though we know we should be. (It does get more complex for multi-issue, multi-facet, screening, etc, but we do that stuff (by hand) differently too.)

posted 01-07-2009 11:39 PM            

let's get one thing understood; I believe, YOU believe OSS is the greatest thing since sliced bread for making decisions after a polygraph exam. I get that! I was a little more specific to your posting comments.

I do not believe 10 "hand-picked" examiners equals "most examiners" or even scientifically sound.

Also, while I didn't read the study or consider specific academics and/or mathematical equations to make a call (because I do not, nor will I use it to make up my mind concerning the test results), I find it hard to understand that an algorithm makes calls more accurately (I believed your word was "outperformed", which I have no idea what that means) 9 out of 10 examiners...

Barry, there are only 3 possibilities in a known truth examination with full compliance, NDI/DI/Inc. So what you're saying is that 9 out of 10 "hand-picked" examiners were "wrong?" Now, I am becoming concerned for the chart evaluation training our national and state associations have been presenting.

I don't know about you, but I hardly have time or inclination to grade 100 charts.

You wrote, "(The Utah algorithm outperformed hand scorers when it was tested too - all the way back in the 80s.)"

Is this the OSS program?

You accurately noted that Polyscore says it requires four charts - not three as was used here. Interestingly, I think that effort was produced long before it became common acceptable practice to run 4 charts, if needed. Meaning, the research was conducted before the test format was acceptable or approved. And while I am not sure 4 charts is anything more than for clarity i.e. if close, it is not always necessary. I wonder (out loud) why Polyscore did that?

You wrote "OSS-3 is free. If anything, they've lost money."

Yes, for now. Nothing is free and NO-ONE puts in the kind of effort you've reported that went into OSS and they're willing to accept nothing but financial losses! Sorry, I'm a realist and in my opinion, while even a crack dealer will give you something for nothing, there remains a reason.

You said OSS had "perfect reliability", then went on to say you have had success (confessions) with opposite calls. I'm not nit-picking, but I just find it very risky to begin placing ALL confidence in an algorithm and ignoring the basics of our profession (i.e. evaluating physiology).

You wrote OSS "measures three features: RLL and amplitude increase in the cardio and EDA." This seems to me to be exclusionary of other diagnostic features currently used, taught and acceptable within chart evaluation. And, while placing more or less emphasis on specific features, I am not sure any algorithm can effectively hone itself to a specific examinee, each and every test. If OSS does, how does it adjust to make that modification and evaluative emphasis that one person is a EDA responder, another respiratory responder, etc?

Thanks for the input. This is interesting.

posted 01-08-2009 12:09 AM            

I am of the opinion that you are both right (mostly).

Jim:

quote:

---

if you, the examiner of record, score it plus 5 at both spots, then it is NDI/NSR.

I believe there is an ethical issue in using or deferring to algorythms for test results. (garbage in/garbage out)!

---

Yep. The examiner has to be responsible for what is going in. At this point, we cannot expect the computer to be capable of scoring anything that you wouldn't score. This may change, but that change will probably be in the form of understand more about why the data are un-interpretable.

Jim:

quote:

---

find it interesting that Polyscore has not been validated against what I believe is the most valid polygraph technique in use today, the Bi-zone/You Phase format.

This being the case, I am interested in knowing what testing formats have Polyscore been validated against? Any? Maybe I need to change my formats...

---

I like the Bi-Zone/You-Phase, tightly focused no-nonsense structure.

To my knowledge Polyscore has not been validated with Bi-Zone tests. However, that doesn't mean that the Polyscore decision model won't apply to Bi-Zone tests – we don't know because there is not enough published documentation on the mathematical transformation models to know. If the transformations allow for a known method to estimate the variance for total and spots when using more/fewer questions or charts, then there is no real theoretical problem applying the algorithm to the BiZone. We don't know, because of proprietary/intellectual-property concerns (ie., fear of criticism or economic loss).

I believe Polyscore was validated using a k-fold technique, and the ZCT case from the Polyscore training sample. K-fold is a really cool way of estimating the confidence intervals and expected bias/shrinkage of the method – its a statistical technique for accomplishing validation goals without a validation sample. Mostly, there is just not quite enough in publication to feel like we know what goes on in Polyscore.

I believe Polyscore was subsequently evaluated with MGQT cases, but I don't know the results – only that they released a version that would score the MGQT, in about 1998.

We do know that other transformation models, such as rank-order transformation models and cumulative transformation models, do not theoretically apply to differing test structures - because the variance of spot and total can be expected to differ. Rank and cumulative models therefore require training and validation with every different technique.

OSS-3 (and all methods) still require validation, but it's easier to accomplish because we have solved the transformation problems in a way that allows us to have known estimates of the variance of total and spot scores regardless of the number of RQs or the number of charts.

It really is all about variance – knowing it and controlling it (or at least estimating and approximating it).

Knowing the variance for spot and total scores, regardless of the number of RQs and charts, makes it theoretically and mathematically justifiable to apply the OSS-3 algorithms to BiZone or other tests, even though it was trained on ZCT exams.

I believe it is ethical to base our professional opinions on what we know and understand. Computers simply do the heavy-lifting and number-crunching for us, and give us a statistical classifier (p-value or probability-value) for our result. At present, we don't know the probability values for our hand-scores, because we haven't adequately studied and published the distributions of our various hand scoring systems. That is a deficiency in the state of our science. We may not think about it, but our scientific minded critics and opponents do. So it is “ethical” to study our scoring methods and begin to describe the level of statistical confidence in our decisions.

It is also ethical to use a computer to do the math, just as it is ethical for the engineer who designed the planetary gears in your automatic transmission to use a computer to calculate the optimal surface angles on those gears. It could be done by hand – just as OSS-3 could be calculated by hand – but why would we want to do that when a computer will do it faster and will fewer decimal and calculation errors. The gears will run smoother and last longer and that mean more time zoomin' down the super-slab at the posted legal limit and less time waiting for a tow. It is also ethical to expect the engineer to fully comprehend and understand the calculation of those angles. It is unethical for a hack/neophyte engineer to attempt to calculate all those gear angles by hands, or to use the computer calculations with no understanding of what the 'puter did.

Another ethical consideration is whether we human-experts actually understand what features the computer algorithms is looking at. Knowing how to see those features ourselves allows us human-experts to essentially QC the work of the algorithm. If we know the features, and the transformation model, and the decision model, and the norms, then we can then understand why the computer came up with what it did. If we don't know these things, then it's a black-box – and out ability to depend on it is reduced.

Barry:

quote:

---

If you read up on OSS-3, then you won't be "blindly" accepting anything, and you'll probably find it'll outperform you if you want to accept a challenge match. Since it does outperform most examiners, there's room for an argument that not using it is unethical.

---

Exactly.

Jim:

quote:

---

Also, I have heard of many examiners who use Polyscore to give them "the answer"; and this, to the exclusion of their own data evaluation - that is, if they even bother to make their own evaluation. This is a very dangerous and slippery slope for our profession.

---

What if someone knows they don't score as well as the computer. Is it more ethical to use the computer or score the data by hand?

OK. I'll concede this one. It is ethical for me to use a computer to score an MMPI-2, as long as I am capable of scoring it by hand and interpreting it myself. If I am not capable or qualified, then feeding MMPI-2 data to a computer and letting the computer diagnose someone is patently unethical.

Examiners have to be able to evaluate their charts themselves.

However, in our experiments (in the current APA journal), OSS-3 outperformed 9 out of 10 experienced human scorers. One person outperformed the algorithm. It wasn't me – and, curiosity aside, I don't want to know who it was.

Jim:

quote:

---

Further, while I have no knowledge or information about OSS (though I have seen all the fanaticism surrounding its introduction and promotion), I do not believe any computer algorithm is more accurate than an examiner (of record) in field use, during actual specific issue testing. Do you have any independent research to support your claim to the contrary? Also, you wrote, "Since it does outperform most examiners", please present your support, as I would find that claim hard to support (though not necessarily difficult to believe in some circumstances/examiners I have witnessed). Additionally, have "most examiners" been tested against it with their existing accuracy and ability? I for one, was never contacted to participate in that experiment/research.

---

Fanatic: (that would be me, I think.)

Barry:

quote:

---

Read your current APA journal.

---

It seems to me to be a most interesting journal – the lead article is a validation of the AFMGQT with 2 RQs, in a single-issue lab scenario.

Barry:

quote:

---

Initially, OSS-3 went up against 10 virtually hand-picked scorers who were believed to be very good at polygraph. (The scorers scored field tests - for paired testing certification -in which ground truth was known.) Of the 10 scorers, OSS-3 outperformed 9, and I don't think there was a significant difference between the top scorer and OSS-3. All you have to do to accept the "challenge" is score 100 of the charts in the paired testing sample. That's been an invitation that's been out to all here for quite some time.

(The Utah algorithm outperformed hand scorers when it was tested too - all the way back in the 80s.)[quote]

We replicated the Utah algorithm – in the current APA journal – it is a formidable and sound method.

Barry:
[quote]In regard to Polyscore, you can read it's very short manual and see what it was validated on. You can use it for a Bi-zone, but Polyscore says it requires four charts - not three as was used here. (Although, I don't think that would make much difference, but who knows?)

---

That is my point (earlier). Its about variance, and the need to know the distribution of scores. The need for four charts gives 8 total RQ presentations, compared to 9 presentations for the 3-chart ZCT. The fact that this is a concern suggests a cumulative data model. Thanks. I don't have Polyscore or the manual.

The OSS-3 solution is to standardize the data – so it doesn't matter how many charts or RQs. We know the distribution – Gaussian, mean = 0, SD = 1, etc. The Utah algorithm also standardizes the data, but in a different was than OSS0-3 – it's described in Kircher & Raksin (1988), Raskin, Kircher Honts & Horowits (1988), and Kircher and Raskin (2002) – and in the Stoelting manual.

Utah first averages and then standardizes the component scores between charts, and then combines the standardized component scores with a discriminate function, using the classical maximum likelihood and linear method that is common to discriminate analysis.

OSS-3 first standardizes the components within charts, then aggregates the within chart component data using weighted averaging and canonical discriminate function, before averaging the between chart results. (breathe). The advantage of our method is that I believe it will be less vulnerable to the measurement noise introduced by between chart differences or changes in tracing sensitivity/amplitude levels.

There is a complete description of the OSS-3 model in the current APA journal.

Be warned though – the journal editor seems to have injected some type of chloroform into the ink.

Barry:

quote:

---

There was an Israeli study on the Backster version of the Bi-zone. It wasn't any more accurate than any other, which has been, Honts, for example, position for some time: a good CQT with a couple of well-developed CQs and a couple of well-developed RQs, then you've got a good CQT that should have an accuracy in the neighborhood of other good CQTs. I haven't read it yet, but there is a study published on the AF MGQT 2-RQ / 3-CQ version in the current edition of Polygraph too.

---

Where is the Israeli study?

Barry:

quote:

---

OSS-3 is free. If anything, they've lost money.

---

We learned a lot, and that was our objective. Perhaps we'll get some karma-points.

We developed it because we wanted an algorithm with a completely documented decision model, which could be validated on the types of tests we actually run. We also wanted a model that could be available cross-platform, which anyone can study and improve, and use to advance our field of science. To do this we had to agree in advance that we are not going to fear any form of scrutiny or criticism, or economic loss for telling anyone as much as they want to know about it. To do that, it seemed to us that it had to be free, and to accomplish that we had to do it without funding. Actually, we knew we had no funding and wanted to do it anyway, so we decided it would be great to do it as an open-source project and give it away.

OSS-3 was developed and validated on no budget (zero, nada, zippo, zilch), and perhaps now close to 2000 hours of “spare time,” since late 2006. The great thing is that lot's of people helped us. We had data from the DACA 2002 archive, and got additional data from Texas, and Colorado. Paul Menges and others had some input into parts, and we've endeavored to cite everything that we got from people like Kircher, Barland, Krapohl, Senter, and others. So, we're really just the construction crew that bolted the pieces together – in science and teaching we call it “scaffolding” because we keep building on the layers of knowledge that were given to us by others.

We didn't just make up OSS-3. It is built on over 20 years of existing published ideas and published studies (what few there actually are), and the good work of the giants in our field of science. It belongs to no-one, and everyone. It's free – as in freedom (do what you want with it), free – as in free speech (say what you want about it), and free – as is free beer (enjoy it, share it, give it away). It's not free – as in “free room and board at Leavenworth” (if you don't like it, then don't use it.)

Barry:

quote:

---

As far as anyone wanting to use an algorithm to determine the result of an exam, there are many sound reasons. For one, you have perfect reliability.

---

It will also give us a statistical classifier for a test result.

Another important thing is that a well developed automated system can actually help us study our hand-score methods – the computer's reliability can help us to better isolate various aspects of the data, to better determine what actually works, and what aspects of the data don't work.

Barry:

quote:

---

OSS-3 is relatively easy to understand. It measures three features: RLL and amplitude increase in the cardio and EDA. All you have to do is make sure the data is good enough to score, and it'll get you the right answer over 90% of the time. (And yes, I've disagreed with it before - and got a confession.) The math would be difficult for many, sure, but all it does is weight the components differently - something hand-scorers don't do even though we know we should be. (It does get more complex for multi-issue, multi-facet, screening, etc, but we do that stuff (by hand) differently too.)

---

With accuracy around 90%, give or take a few percentage points up or down, we're all going to disagree with it at times. This will be the case for any method.

Jim said it best...

quote:

---

but polygraph instrumentation (after 100 years) is still three physiological channels and looking for "bumps"...

---

Which bump is bigger? (red or green)

Mark Handler was asking this, and we turned it into a simplified hand-scoring experiment – in the current APA journal. We had inexperienced scorers (students who had not completed polygraph school) score the same cases as the OSS-3 and the experienced examiners – using the bigger-is-better rule and three position scale only.

Niters.

r

posted 01-08-2009 08:13 AM            

You wrote, "What if someone knows they don't score as well as the computer. Is it more ethical to use the computer or score the data by hand?" This is the issue on the table. I am not arguing OSS's ability, I am sure it is a fine program and accurate too.

You stated, "Examiners have to be able to evaluate their charts themselves." That IS my point! And if any examiner knows they can not accurately score charts then it is up to them to get the refresher training, and keep up to date with current rules and professional guidelines, etc.

If they do not, then I consider it a lapse in professional judgment and ethical responsibility by the examiner when they "must" rely solely on computer algorithms for determinations of findings.

Nothing more pathetic than an examiner being asked, in court, to re-score his charts and not being able to because they don't have their computer algorithm handy...(BTW, this has happened).

Thanks,

Jim

P.S. I was thinking. What if an examiner made a call using only an algorithm with no hand scoring effected (because they can't or are too lazy, whatever...). What if that call was wrong and the examiner was sued. Do the professional organizations we belong to support that examiner's action and will they send a representative into court to defend me? What of Error and Omission Insurance? Will Melanie's company defend me too? Just an afterthought for consideration.

posted 01-08-2009 09:35 AM            

quote:

---

You stated, "Examiners have to be able to evaluate their charts themselves." That IS my point! And if any examiner knows they can not accurately score charts then it is up to them to get the refresher training, and keep up to date with current rules and professional guidelines, etc.

If they do not, then I consider it a lapse in professional judgment and ethical responsibility by the examiner when they "must" rely solely on computer algorithms for determinations of findings.

---

Agreed.

quote:

---

let's get one thing understood; I believe, YOU believe OSS is the greatest thing since sliced bread for making decisions after a polygraph exam. I get that! I was a little more specific to your posting comments.

---

Wrong. You missed it. I was trying to stir a quiet forum. I seldom use OSS-3 as I use CPS, which doesn't have it yet.

posted 01-08-2009 09:59 AM            

I'm glad (I think) that you haven’t changed over the years.

If you get a chance, I would suggest that you take the time and score those 100 charts. It takes a little time, but over several days or weeks it can be done w/o much inconvenience. They are known truth charts and it allows you to do a self evaluation, if nothing else, of your hand scoring abilities. I think it is worth the time and effort.

posted 01-08-2009 11:30 AM            

That's about the best solution for now.

What is bothersome to me is when we observe a difference in our scores and the algorithm scores.

But keep in mind that algorithms and math are not magic and not witchcraft. There is a reason the results come out the way they do.

If we have access to information that allows us to understand what the computer does, then we can understand the causal chain of events that makes the results occur the way they do. We can also adapt and improve, and do a better job being in control of the situation and accounting for our work.

Just as we use the algorithm to CQ ourselves, we should use our own expertise to QC the algorithm.

We can't do that without access to the information, and that means we have to open the black box.

----------------------

We spend 4.5 billion on an aircraft carrier. Do we want the captain to navigate with or without GPS? Probably with GPS. Do we want the captain to be able to navigate without GPS? You betcha.

--------------------

If we think that the polygraph, as it is today, is good enough, and has achieved everything it can possibly ever achieve, then we have no need to learn or do anything new.

If not, then we need to keep learning.

Continued learning means we need to be thoughtful about whether we believe yesterdays solutions will solve today's and tomorrow's problems. They may not. History doesn't support that notion. Fortresses that were somewhat effective against catapult assaults gave way to improved artillery, and by the Civil War we stopped building fortresses in favor of better cover at or below ground level. Napoleonic lines of infantry don't work any more, and the idea of battle-lines is becoming arcane. Today we navigate with the aide of satellites more often than with stars.

----------------------

We have this same dilemma in PCSOT right now.

Do we think that solutions from the 1998 JPCOT era, local to Texas, are sufficient to address contemporary problems at the national/cross-jurisdiction level?

Probably not. We need evolved solutions for evolved problems.

-------------------------------

Do we think we are going to advance the accuracy of polygraph science, or win arguments with our scientific minded opponents, by limiting our repertoire to math that my son learned in the 2nd grade?

Probably not.

If we want a better polygraph, then we can't stay locked into old solutions. We have to seek better data, better testing procedures, better scoring/interpretive procedures, and
better decision models.

There is quite a bit that we could learn and incorporate about decision theory, if we had the time and inclination. But it requires that we have the courage to challenge our present mental models and procedures.

Or we could limit ourselves to procedures and solutions that were adequate 10 or 20 or more years ago.

--------------------

Now's about time to see those charts...

What we are really arguing is how to integrate the results of computer algorithms into our professional judgment and repertoire.

There is a reason for these things, and it can be studied. But not without access to the information. Without the information, we're just guessing and hypothisizing - which is really just blowing smoke at each other.

Peace,

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 01-08-2009 02:37 PM            

My concern, as you might guess, is that I don't see how scoring the 100 ground truth charts tells us much about whether or not the person scoring them can conduct a polygraph and might be misleading about the examiner's ability to score charts.

I guess if the charts were scored by a large group of examiners and a clear majority called a set as deceptive rather than non-deceptive, oppostite the actual results, then are all the examiner wrong or are the charts (and therefore the polygraph)wrong?

Also is the requirement to get the correct condition (NDI/NDI/NO)? If you call a set NO is that an error or a no call.

posted 01-08-2009 06:05 PM            

You ask some good questions, which have been subject to debate. There are likely some charts in the database that will score NDI or DI (to a vast majority of examiners), but in fact won't be correct decisions. Personally, I don't think you should test scoring ability on "bad" charts. The flip side argument is that it's real life so we shouldn't play with them. (I think Ray might have looked and found a few of them. I don't think anybody else ever did.)

NO / INC calls are not counted as errors. You can have up to 20% errors and meet the certification requirements.

What we found from those scoring exercises is that most examiners can score DI charts, but they weren't very good at calling the NDI. I think that gives our detractors ammo - too much ammo. They say, "See, polygraph doesn't work with the truthful." The reality is that the examiners are just not good at scoring.

Again, if a algorithm can get 90% correct and and examiner only 50% (some were worse), then the data is there. We need to better train our examiners. The only problem is that everybody thinks he's the best scorer there is and doesn't need to test him or herself. (Egos in this business? Say it isn't so.)

Ray,

posted 01-08-2009 07:23 PM            

My use of the term "ground truth" is, I believe, incorrect. I think they are confirmed DI or NDI. I think that is why they use those charts, because they know exactly how they should be scored.

I don't view a no opinion as a error. Obviously the examinee was either being truthful, or he was not. A no opinion means I don't have enough data to make a call. Our opponents like to call a no opinion an error.....I disagree.